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SYSTEM AND METHOD FOR VTOEO CONTENT ANALYSIS-BASED 
DETECTION, SmVEILLANCE, AND AIA^ 

BACKGROUND OF IHE INVENTION 
5 RELAIED APPLICATIONS 

The present invention relates and claims priority &om US provisional 
patent application serial number 60/354^09 titled ALARM SYSTEM BASED 
. ON VIDEO ANALYSIS, ffled 6 February 2002. 

10 FIELD OF THE INVENTION 

The inresent invration relates to video surveillance systems in general, 
and more particularly to a video content analysis-based detection, survdllance 
and alarm management system. 

15 DISCUSSION OF THE RELAIED ART 

Due to the increasing number of terror attacks and potential tmror- 
related threats, one of the most critical surveillance challenges today is the timely 
and accurate detection of suspicious objects, such as unattended luggage, illegally 
parlced vehicles, suspicious persons, and the like, in or near airports, train stations, 

20 federal and state government buildings, hotels, schools, crowded public places 
typically situated at city centers, and other sensitive areas. In accordance vAftk the 
prevailing known tactics of t^orist organizations, imattended innocent-looking 
objects, such as a suitcase, could contain hidden e^losive materials installed 
therein to effect a controlled estplosion and thereby inflict noassive inq>act damage 

25 to the near enviroiimeait and the individuals within. Likewise recent attacks have 
been p^petrated througih the use of vehicle bombs, semingly innocent until 
explosion. Since these hidden explosives afe usually activated by the setting of 
carefully timed (typically short-period) detonator means, or by an operator v/bo is 
at tibie scene or close by, the prompt, rapid and timely detection of suspicious 

30 objects, such as unattended luggage, vehicles paridng in forfoiddea zones, 

1 
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, suspicious persons, persons leaving unattended suitcases or vehicles, and the like, 
could prevent life-tbreatening situations. Similarly, it is important in aieas, such 
as airports, to be able to track persons and objects, such as suitcases and cars, to 
assist in locating lost luggage, and to restrict access of persons or cars to obtain 

5 zones. The applications of such abilities are not only for security pmposes. 

Recently, the authorities responsible for the safely of the public have 
been attempting to cope with the problems listed above in tiie most obvious 
manner by increasing the number of human personnel tasked for the detection, 
identification and consequent handling of suspicious objects, including vehicles, 

10 luggage and persons. At tiie same time, in order to maintain substantially 
unobstmcted passenger flow and in order to minimize transport delq^ and 
consequent public frustration, the security personnel have been obliged to utilize 
inefficient and time-consuming procedures. One drawbadc of flie above human- 
centric solution concerns the substantially increased expenses associated with the 

15 hiring of a large number of additional personnel. Another drawback concerns the 
inherent inefficiency of the human-centric procedures involved. For example, 
specific airport security personnel must perform visual scanning, tracking, and 
optional handling of objccts in sensitive transit areas 24 hours a day, where the 
sheer number of luggage passing tbrou^ these areas effect inaieased &tigue 

20 accompanied naturally vnSi diminished concentration. Jn &e same manner, in a 
traffic-extensive area wherein specific security personnel must watch, track and 
optionally handle vehicles parking m restricted areas, natural weariness soon sets 
in and the efficiency of the human-centric procedure gradually deteriorates. 

Current^ available siureillance sfystems are designed for assisting 

25 human security officers. These systems typically mclude various image 
acquisition devices, such as video cameras, for capturing and recording imagoy 
content, and various detector devices, such as movemait detectors. The existing 
surveillance systems have several important disadvantages. The type of alarms 
provided by the detectors is substantially limited The video images recorded by 

30 the cameras sure required to be nionitored constantly by human security personnel 

2 
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in order to detect suspicious objects, people and bdiavior. An alann situation has 
to be identified and suitably handled by the personnel where a typical handling 
activity is the manual generation and distribution of a suitable alarm sigoal. Since 
these surveillance systems are based on human int^ention the problems related 
5 to -natural human-specific processes, such as fatigue, lack of concentration, and 
Ifae like, are still remain in effect 

A further drawback of existing surveillance systems concern the 
failure of those systems to handle certain inherently suspicious events that w^ 
captured by the cameras monitoring a scene. For example, current surveillance 

10 .systems associated with airport security application, typically &il to id^tify a 
situation as suspicious wh^ the situatioin involves a vehicle arriving at a 
monitOTcd airport terminal, an occupant of the vehicle leaving of the vehide, and 
tiie departure of the occupant fix)m the monitored scene in a direction that is 
opposite to the tenninaL 

15 Yet another drawback of the current systems concon the inability of 

the current systems to identify a set of events linked to the same object in the 
same area throughout a pre-defined surveillance period. For exan^le, when 
"suitcase" object is left in the scene by a first person and later it is picked up by a 
second person tiien the leaving of the suitcase and the picking up of the suitcase 

20 constitute a set of linked events. 

Still another drawback of the current systems concern the inh^^t 
passivity of systems due to the fact tiiat the operations of the systems are based on 
events initiated by the operators and due to the fact that the systems provide no 
built-in alerts. 

25 In addition, existing systems are incapable of associating a retrieved 

cvGot or object through the use of important parameters, sudi as color of hair^ 
color of clothing and shoes, complexion (via the use of a color histogram), facial 
features (via face recognition routines), normalized size of the object (distance 
from the camera), and the like. 
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It would be easily perceived by one with ordinary skills in the art that 
tiiere is a need for an advanced and enhanced surveillance, object tracking and 
identification system. Such a system would preferably automate the procedure 
concerning the identification of an unattended object substantially and would 
5 utilize cost-effective, efficirat methods. 

SUMMARY OF Tfffi PRESENT INVE^ 
One aspect of the present invention relates to a mettiod for analyzing video 
data, comprising receiving a video frame, comparing said video frame to 

10 background reference fi^e to locate difference, locating a plurality of objects to 
form a plurality of marked objects; and detennining a behavior pattern for an 
object according to the difference, said behavior pattern is defined according at 
least one scene characteristic. The method further comprises producing an 
updated background reference fi^e. The method finlfaer comprising 

15 detennining the difference performed by creating a difference fi-ame between the 
video fi-ame and the backgroimd reference frame. The method further coniprises 
finding a new object whra detennining the difference and an alarm accordmg to 
said behavior pattern. A pre-4efined pattern of suspicious behavior comprises an 
object presenti ng unpredi ctable bdiavior. 

20 A second aspect of the present invention relates to a system for analyzing 

video data comprising a plurality of video firames, the system comprising, a video 
frame preprocessing layer for determining a difference between a pliirality of 
video firames, an object clustering layer for detecting a plurality of objects 
according to said differoice, and an application layer for characterizing said 

25 plurality of objects according to scene characteristic. The difference is 
determined between a video frame and a reference frame. The system finlher 
comprises a backgroimd refreshing layer for preparing an iq)dated reference 
fi^e according to the said difference. The scene diaracteristic define a 
behavior pattem for an object, such tiiat if the object exhibits tiie behavior 

30 pattern^ the scene characteristic is detected. If the scene diaracteristic is detected. 
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an alann is generated. The sceae characteristic further coniprises a paranoid for 
detennining if the object exMbits the bdbavior pattens 

A third aspect of the present invention refers to a system for detecting a 
vehicle remaining in a restricted zone for at least a minim um period of time, 
5 comprising, a video content analysis module for analyang video data of the 
restricted zone, said video content analysis module further comprising an object 
tracking component, and an application layer for receiving data fiom said video 
contattt analysis module and for detecting a vehicle remaining in the restricted 
zone for the minimum period of time, and said application layer generating an 

10 alarm upon detection. 

A fourth aspect of the present invention refers to a system for detecting 
unattended luggage, bag or any imattended object in an area, comprising, a video 
content analysis module for analyzing video data of the area, said video content 
analysis module fmlfaer comprising an object tracking component, and an 

IS application layer for receiving data from flie video contrat analysis module and 
for detecting an imattended object, wherein said uiiattended object has not been 
attended in the area for more then a predefined period of time. 

A fifth aspect of the present invention refers to a surveillance system for the 
detection of an alarm situation, the i^stem comprising the elraients oj^ a video 

20 analysis unit for analymig video data representing images of a monitored area, 
the video analysis unit comprising an object tracking module, to track the 
movemmts and tiie location of a video object, a detection, surveillance and alarm 
application for receiving video data analysis results firom the video analysis unit, 
for identifying an alarm situation and to generate an alarm signal, an events 

25 database to hold video objects, video object parameters and evoits identified by 
the applicatioiL The system comprises the elem^ts o^ an application driver to 
control the detection, surveillance and alarm application, a database handler to 
access, to update and to.read the events database, a user inter&ce component to 
c ommimi c at e with a user of the system, an application setap and control 

30 component to define the control parameters of tiie application, an application 
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setap parameters table to store the control parameters of the application. Vas 
systm further comprises the elements o^ a video data recording and 
compression unit to record and compress video data representing images of a 
monitored area, a video archive file to hold the recorded and compressed video 
data representing unages of tiie a monitored area, an alarm distribution unit to 
distribute llie alarm signal representing an alarm situation. Hie system fiaiher 
comprises the elements o^ a video camera to obtain the images of a monitored 
area, a video capture component to capture video data representative of the 
images of the momtored area, a video fransfer component to transfer the captuied 
video data to the video analysis unit and the recording conqjressing and ardiiving 
unit, a computing and storage device. The object traddng module comprises the 
elemaits ot a video frame preprocessing layer for detennining flie difBaeoce 
between video frames, an objects clustering layer for detecting objects in 
accordance with the determined difference, a scene characterization 1^ for 
15 characterizing the object according to characteristic of a scene, a background 
refreshing layo: for preparing an updated reference according to the detennined 
difference. The detection surveillance and alarm appUcation is operative in the 
detection of an unattended object in the momtored area. Any video camera wi&in 

tiie system, the video capturing cotapoaeat, the video tr^isfcr conq)onent and the 
computing and storage device caai be separated and can be located in different 
locations. The interfece between the video camisia, the video capturing 
component, the video transfer component and tiie computing and storage device 
is a local or wide area network or a packed-based or ceUular or radio frequency 
or micro wave or sateUite network. The unattended object is a luggagp left m an 
25 airport terminal for a pre-determined period or a vdiicle parking hi a restricted 

zone for a pre-defined period. The detection surveillance and alarm q)pUcation is 
operative in the detection of an unpredicted object movement The analysis is 
also performed on audio data or thermal imaging data or radio frequency data 
associated witii tiie video data or the video object in synchronization witii the 
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video data. The video capture component captures audio or thermal infomiation 
or radio frequency information in synchronization with the video data. 

A sixth aspect of the present invention ref^ to a surveillance method for 
the detection of an alarm situation, the surveillance to be performed on a 
5 monitored scene having a camera, the method comprising the steps o^ obtaining 
video data from the camera representing images of a monitored scene, analyzing 
the obtained video data representing images of the object wifliin the monitored 
scrae, the analyzing step comprising of identifying the object within the video 
data, and inserting the identified object and the event into an event database. 

10 Another embodiment of the method fiirtii^ comprises the steps o^ retrieving of 
tiie object associated with an evont, according to user instroction displaying the 
video event associated with the retrieved object The method further comprises 
the steps ofi retrieving at least two events, associating according to parameters of 
the object, the object with tiie at least two events. The method comprises tiie 

15 steps o^ debriefing the object associated with tiie event to identify the pattern of 
behavior or movement of the object witiiin the scene within a predefined period 
of time,. _Ilie method furtiier comprises the steps of, pre-defining patterns of 
suspicious behavior; and pre-defining control parameters. Hie method further 
comprises the steps o^ recognizing an alarm situation according to the pre- 

20 defined patterns of suspicious behavior, and g^erating an alarm signal 
associated with the recognized alarm situation. The method further conopises tiie 
steps oi^ implmenting patterns of suspicious behavior introducing pr&*defined 
control parameters, recording, compressing and archiving the obtained video 
data, distributing the alarm signal representing an alarm situation am>ss a pre- 

25 defined range of user devices. The pre-defined pattacn of suspicious behavior 
comprises, an object entering a monitored scene, the object separating into a first 
distinct object and a second distinct object in tiie monitored scene, the first 
distinct object remaining in the monitored scene without movatn^ for a pre- 
defined period, and the second distinct object leaving the monitored scene. The 

30 pre-defined pattern of suspicious behavior comprises, an object -entering the 
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monitored scoie, the object ceasii^ its movemrait, the size of the olgect is 
recognized as being above a pre:defined parameter value, and the object 
remaining immobile for a paiod recognized as being above a pre-defined 
parameter value. Hie method does further comprise identifying information 
5 associated -with flie object for the purpose of identifying objects. 

BRIEF DESCRIPTION OF TOE DRAWINGS 
The present invention will be understood and q)preciated more fidly 
from the following detailed description taken in conjunction with the drawings in 
10 which: 

Fig. 1 is a schematic blodc diagram of the system architecture, in ' 
accordance with the preferred embodiments of the present inventiQI^ 

Fig. .2 is a sunplified flowchart that illustrates the operation of the 
object tracking method, in accordance with the preferred embodiment of the 
15 present mvention; 

— _ Fig. 3A is a simplified flowchart illustrative of the operation of 
unattended luggage detection application, in accordance with the first preferred 
embodiment of the present inventiQE^ 

F ig. 3B illustrates the control parameters of the unattended lu^^e 
20 detection application, in accordance with the first preferred embodiment of the 
present invention; 

Fig. 4A is a sunplified flowchart illustrative of the operation of the city 
centCT application, in accordance with the second preferred embodiment of the 
present invention; 

25 Fig. 4B illustrates the control parameters of the chy center application, 

in accordance with the second preferred embodiment of the present inv^ition; and 
Fig. 5 is a flowchart describing the operation of the proposed method, 
in accordance with the prefixed embodiments of the present invention. 

30 
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. DETAILED DESCRIPTION OF TEffi PRE 

A monitoring and surveillance system and method for Ifae detection of 
potential alarm situation via a recorded surveillance content analysis and for the 
management of the detected unattended object situation via an alarm distribution 

5 mechanism is disclosed. The proposed system and method includes an advanced 
architecture and a novel technology operative in capturing surveillance contmt, 
analyzing the captured content and providing in real time a set of alarm messages 
to a set of diverse devices. The analysis of the captured content comprises a 
imique algorithm to detect, to count and to track objects embedded in the captured 

10 content The present invention provides a detailed description of the applications 
of this method The method and syst^ of the preset invmtion may be 
implemented in the contejct of unattended objects (such as luggage, vehicles or. 
persons), parking or driving in restricted zones, controlling access of persons into 
restricted zones, preventing loss of objects such as luggage or persons and 

15 counting of persons. 

In the preferred embodiments of the present invention, the monitored 
content is a video stream recorded by video cameras, captured and sampled by a 
video capture device and transferred to a video processing unit The video 
processing unit p^orms a content analysis of the video images and indicates an 

20 alarm situation m accordance witii the results of the analysis. In other preferred 
embodim^t of tiie invention, div^e other content formats are also analyzed, 
such as thermal based sensor cameras, audio, wireless linked camera, data 
produced from motion detectors, and the like. 

25 . Hie first preferred embodiment of the preset invmtion concerns the 

detection of unattended objects,' such as luggage in a dynamic object-ridi 
environmrat, sudi as an airport or city center. The second preferred embodiment 
of the invention concerns the detection of a vehicle parked in a forbidden zone^ or 
the extended-paiod presence of a non-moving vehicle in a restricted-pmod 

30 parking zone. Forbidden or restricted parking zones are typically associated with 
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sensitive trafBc-intensive locations, such as a city center. Another preferred 
embodiment of the invention concerns the tracking of objects such as persons in 
various scenarios, such as a person leaving the vehicle away firom the terminal, 
which may equal suspicious (unpredicted) behavioral pattern: In other possible 
5 embodiments of the present invention the system and me&od can be impl^ented 
to assist in locating lost luggage and to restrict access of persons or vehicles to 
certain zones. Oth^ preferred embodiments of tiie invention could regard the 
detection of diverse other objects in diverse other environments. The following 
description is not meant to be limiting and the scope of the invffltion is defined 
10 only by the attadbed claims. 

Refening to Fig. 1 a set of video cameras 12, 14, 16, 18 operate in a 
security-wise sensitive environment and cover a specific pre-defined zone that is 
required to be monitored. The area monitored can be any area preferably in a 

15 transportation area including an airport, a city center, a building, and restricted or 
non-restricted, areas within buildings or outdoors. The cameras 12, 14, 16, 18 
could be analog devices or digital devices. The cameras can capture normal li^t, 
inj&a-red, temperature, or any other form of radiation. By usmg audio capturing 
devices such as a microphone (not shown), the cameras can also capture auditoiy 

20 signals, such as noise generated by machines and voices made by persons. The 
cameras 12, 14, 16, 18 continuously acquire and transmit sequences of video 
images to a display device 21, such as a video terminal, operated by a human 
operator. The display device 21 could be optionally provided with video images 
fi^om the vide archives 40 by the computing and storage device 24. The cameras, 

25 12, 14, 16, 18 transmit sequraces of video images to a video capture component 
20 via suitabfy wired connections. The video capture componrat 20 could capture 
&e images tfarougjh an analog interface, a digital inter&ce or Ihrou^ aLocal Area 
Network (LAN) interface or Wide Area Network (WAN), IP, T^^eless, Satellite 
connectivity. The video capture component 20 can be a NICEVISION syst^ 

30 manufactured by Nice Systrais Ltd., Raanana, IsraeL The video capture 

10 
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componeat 20 can also be configured to capture audio signals c^tured by the 
- cameras audio capturing devices. The audio and video* signals are preferably 
synchronized at the system level. The component 20 receives the sequCTces of 
video images and appropriately samples the video stream. Where the processing 

5 of the captured video stream is performed by an external computing platform, 
sudbt as a Personal Computer (PC), a UNIX workstation, or a mainframe 
computer, the unit 20 sends the sampled video information to a video transfer 
component 22. The video transfer component 22 transfers the video information 
to a computing and storage device 24. The device 24 could be an external 

10 computing platform, such as a personal computer (PC), a UNIX workstation or a 
. mainframe computer having appropriate processing and storage units or a 
dedicated hardware such as a DSP based platform. It is cont^plated that future 
hand held devices will be pow^M enough to also implement device 24 fliere 
within. The device 24 could be also an array of integrated circuits witii built-in 

15 digital signal processing (DSP) and storage capabilities attached directly to the 
video capture component -20. The capture component 20 and tiie- transfer 
component 22 are preferably separate due to the &ct that a cq>ture component 
can be located at the monitored scene, while a transfer component can be located 
away from the monitored scene. In another preferred embodiment tiie cq)ture and . 

20 the transfer components can be located in the same device. The device 24 
includes a video analysis unit 26, an application driver 30, a database handler 32, 
a user inter&ce 34, a setup and control Detection Surveillance and Alarm (DSA) 
application 36, a recording compressing and archiving unit 38, a video archives 
40, an evmts database 42, an alarm distribution unit 44, a DSA application and 

25 seti^ parameters file 46, and DSA application 48. Optionally, whmever video is 
cq)tured and processed, audio signals captured in association with these captured 
video signals can be stored and tagged as relating to the video c^tured and 
processed. 

Still referring to Fig. 1 the video content is transfOTed optionally to the 
30 recording, compressing, and archiving unit 38. The unit 38 optionaUy compresses 



11 



wo 03/067360 



PCT/iL02/01042 



the video conteot and stores fhe con^ressed content to the video ardiive files 40. 
The video archive files 40 could be suitable auxiliary storage devices, such as 

optical disks, magnetic disks, magnetic tapes, or the like. The stored content is 
held on the file 40 for a pre-defined (typically long) period of time in order to 

5 enable re-play, historical analysis, and the like. In parallel the video content is 
transferred to the video analysis unit 26. The unit 26 receives the video input, 
activates the object tracking module 28 and activates the application driver 30. In 
accordance, with results of the video analysis performed in conjunction with the 
object tracking module 28, the video analysis unit 26 further generates the 

10 appropriate alarm or indication signal where a specific alarm situation is detected. 
The application driver 30 includes the logic module of the application 48. The 
driver 30 receives event data and alarm data from the video analysis unit 26 and 
inserts the event data and the alarm data via the database . handler 32 into the 
events database 42. .The driver 30 fiirttier controls the operation of the DSA 

15 application 48. The setiq) and control DSA application 36 is used by the user of 
the system in ord^ paibrm system setup, to define control parameters, and tiie 
like* The user interface 34 is responsible for the communication with the user. 
The event database 42 stores the event data and the alarm data generated by the 
video analysis unit 26. The event database 42 also holds the search parameters for 

20 searching objects or evrats for the purpose of investigating the eyoats or objects. 
The search parameters include the object circle-like shape and object location 
paramet^. Oth^ object search parameters can also include data collected from 
various cameras, which may have captured the same object The collected data 
could provide unportant information about the object, such as object type 

25 (ammate or inanimate), object identification (via fece recognition), color 
histogram (color of hair, of cloth, of shoes, of con5)lejdon), and the like. The 
parameters allow fijiding associations between objects and events' captured by 
diflferrat cameras. The alarm distribution unit 44 optionally distributes the 
received alarm signals to a variety of alarm and messaging device. The DSA 

30 application and setiq) parameter ' file 46 stores the setup information and 

12 
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parameters generated by the user via tiie seiup and control DSA plication 36. 
The DSA application 48 provides real-time video to the user, perfirans re-plays of 
video by request, submits queries to tiie event database 42 and provides alarm 
messages, such' as suitably structured pop-up windows, to the uso: via &e uso: 
5 interfece 34. 

Still refoocing to Fig. 1 the units and components desaribed could be 
installed in distinct devices distributed randomly across a Local Area Network 
(LAN) that could communicate over the LAN infrastructure or across Wide Area 
Networks (WAN). Onie example is a Radio Frequaacy Camera fliat transmits 
10 composite video remotely to a receiving station, the receiving station can be 
connected to other components of the system via a network or directly. The units 
and components described could be installed in distinct devices distributed 
randomly across very wide area networks such as the Intacnet Various means of 
communicatian between the constituent parts of the system can be used. Such can 
15 be a data communication network, which cm be connected via landlines or 
ceUular or-like communication derwces and that can be implemented via TCP/ff 
protocols and like protocols. Other protocols and methods of communications, 
such as cellular, satellite, low band, and high band communications networics and 
devices will readily be useM in the implementation of toe present invention. The 
20 components could be further co-located on toe same computing platform or 
distributed across sevoal platforms for load balandng. Tlie conqponents could be 
redundantiy rq)licafcd across several computing platforms for specie 
operational purposes, such as being used as back-iq) systems in the case of 
equipment failure, and the like. Although on the drawing under discussion only a 
25 lunited set of cameras and only a single conq)uting and ston^e device are shown 
it will be readily pea^eived that in a realistic aovmnunent a plurality of cameras 
could be connected to a plurality of computing and storage deWoes. 

Referring now to Fig. 2 showmg illustrates the operation of tiie object 
traddng method. The proposed system and me&od is based upon a video content 
30 analysis metiiod that can detect, track and count objects in real time in accordance 
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with the results of the video stream processing. This mefliod eoables the detection 
of new objects , created from Ihe object identified and tracked. The method also 
enables to identify when an object tracked has merged with another object The 
merging of objects with other objects and the creation of objects from other 
5 objects is particularly important to identify persons leaving vehicles, luggage, or 
other objects in the enviromnent monitored. The ability to detect if an object is 
o-eated or disappears also enables the method of the present invention to identify 
if persons disturb objects or move objects. The method is implemented in liie 
object tracking module 28 of Fig. 1, The method receives the following input 
10 new video frame, background (reference frame), detected objects from the last 
iteration. The outputs of tiie method comprise the updated background, and tiie 
updated objects. Fig. 2 illustrates tiie four layers that jointiy inq)lement object 
tracking and detection method. The video frame pre-processing liqrer 52 uses a 
new frame and one or more reference frame for generating a difference fi?me 
15 representing the difference between the new frame and the reference frame or 
frames. The reference frame can be obtained from one of the capture devices 
described in association witii Fig. 1 or provided by the user. The difference frame 
can be filtered or smootiiened. The objects clustering layer 54 generates 
new^pdated objects from tiie difference frame and flie last known objects. The 
20 scene characterization layer 56 uses tiie objects from the objects clustering layer 
54 in order to describe tiie scene. The background-refireshing hgrar 58 updates tiie 
background (reference frame or frames) for the next fiame calculation and a 
refreshing process uses tiie outputs of all tiie previous layers to generate a new 
reference layec or layers. Note should be taken that in otiier prefared 
25 embodiments of tiie invention ptiier similar or different processes could be used 
to accomplish tiie underlying objectives of tiie system and mettiod proposed by 
the present invention. 

The first preferred embodiment of tiie mvention regards an unattended 
object detection system and metiiod. The unattended object could be a suitcase, a 
30 carrier bag, a backpack, or any oflia: object tiiat was left unattended m a security- 
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sensitive area, such as an airport terminal, a train station's waiting room, a public 
building, or tte like. 

Referring now to Fig. 3A showing a flowchart illustrative of the 
operation of unattended luggage detection application of the method of the 
5 present inventioiL An unattended luggage event is detected via tiie performance 
of a sequence of operative steps. The logic underlying the performance is based 
on a specific scenario. In Ifae scenario it is assumed that a terrorist or any other 
individual having criminal intent may enter a security-sensitive area carrying a 
suitcase. In one example, the suitcase may contain a concealed explosive device. 

10 In another example, the suitcase may have been lost or unattended for a 
prolonged length of time. In yet another application, a suitcase or an object may 
have been taken without authority. In the first example, the individual may 
surreptitiously (in a manna: that non-recognizable by the monitoring cameras) 
activate a time-delay fuse mechanism connected to the explosive device that is 

15 operative in the timed detonation of the device. Subsequently, the individual may 
abandon the suitcase unattended and leave the security-sensitive area for his own 
safety in order not to be e?q)osed to the damage effected by flie expected 
detonation of the explosive devicfe. _ The. . foUowiiig. joperative^ conclusions, 
indicating certain sub-events, are reached by the suitable execution of sets of 

20 computer instructions embedded within a specifically developed computer 
program. The program is operative in the analysis of a sequ^ice of video nnages 
received from a video camera covering a security-sensitive area, referred herein 
below to as the video scene. An unattended luggage evaot is idotitified flie 
program when the foUovring sequence of sub-events takes place and detected by 

25 the execution of the program: a) an object enters the video scene (62). It is 
assumed' that die object is a combined object comprising an individual and a 
suitciase where the individual carries the suitcase, b) The combined object is 
separated into a first separate object and a second separate object (64). It is 
assumed that the individual (second object) leaves the suitcase (first object) on 

30 . the floor, a bench, or the like, c) The first object remains in the video scene 
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without movement for a pre-defined period of time (66). It is assumed that the 
suitcase (first object) was left unattended, d) The second object exits the video 
scene (68). It is assumed that tiie individual (second object) left the video scene 
without the suitcase (first object) and is now about leave the wider area around 
, 5 . the video scene. Following the identification of the previous sub-events, referred 
to collectively as the video scene characteristics, the evCTt will be identified by 
the system as a situation in which an unattraided suitcase was left in the security- 
sensitive area. Thus, the unattended suitcase will be considered as a suspidous 
object Consequently, the proposed system may generate, display and/or 
10 distribute an alarm indication. Likewise, in an alternative embodiment in stop 62, 
a first object, such as a suitcase or person monitored is alrea^ present and 
monitored within tlie video scene. Such object can be lost luggage located within 
the airport Such object can be a parson monitored. In step 64 the object merges 
into a second object Hie second object can be a person picking vtp the lu^age, 
15 another person to whom the first person joins or a vehicle to which the first 
person enters. In step 66 the first object-(now merged with the second object) 
moves firom its original position and. at step 68 of the alternative ^bodiment 
exists the scene. The system of the present invention will provide an indication to 
a human operator. The mdication may be oral, visual or written, llie indicatilon 
20 may be provided visually to a screen or delivered via communication networks to 
officers located at the scene or to ofF-premises or via dry contact to an external 
device such as a siren, a bell, a flashing or revolving li^t and the like. 

Referring now to Fig. 3A which illustrates the control parameters of 
Ibe unattended luggage detection application. In order to set up the unattended 
25 luggage detection application, the user is provided with the capability of defining 
the following control parameters: a) area or areas within the scanned zone 
wherein the system will search for suspected objects (70),. b) the dimensional 
limits of the detected object (72). Objects having dimensions out of the limits 
defined will not be detected as suspected objects, and c) a time out value that is 
30 the amoxmt of time that should pass firom the point-in-time at whidi the suspected 
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object was detected as non-moving and tbe point-inrtime untU an alarm is 
generated. Once an alann is raised the officer reviewing the monitored scene may 
request Ihe^stem to provide a playback so as to identify tiie objects in question. 
Once playback resumes the. officer may tap on a touch sensitive screen (or select 
5 the image by other means such as a mouse, a keyboard, a Ught pen and the like) 
and the system Toay play back the history of video captured in association with 
the relevant object or objects. If a second object, such as a luggage left unattended 
is played back, the playback will identify the person taking or leaving die object 

• The officer may select such person and request play back or forward play to 
10 ascertain where tiie person came from or where the person went to and 
appropriately alert security officers. In order to debrief the event the officer may 
mine the database in various ways. One example would be to request the system 
to retrieve the events or objects that are similar to seardi parameters associated 
with the object or event he is . investigating that can* help in identtfying the 
15 location of person or objects or Ihe whereabouts or actions performed by the 
object A suspect may place a suitcase given to him previously (not in the same 
scene) by a third person and leave the akport in a vehicle. Once the officer is 
alerted to the feet that the Mteas^ is unattended he m^ investigate the retrieve 

tiie tiiird party assodated witii tiie handing of tiie suitcase and tiie vehicle 
20 associated witii tiie suspect Hie system, in real time, stores from each camera tiie 
various objects viewed. In tiie ordmary course of events tiie system would 
associate between like objects captured by various cameras usmg tiie initial 
search parameter (such as tiie suspect's parameters). During tiie mvestigation 
stage tiie system retrieves tiie associated objects and tiie event Imked tiierevwth 
25 and presents tiie events and object to tiie vievring officer m accordance witii his 
instructions. The officer may decide to review forward or backward in time 
scenes the system would marie tiie associated objects flius anowmg tiie officer to 
identify tiie stream of events elected for a particular object, such as existing a 
vdiicle, handing over a suitcase, leaving a suitcase unattended, walking in 
30 uiq>redicted directions and the like. 
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As noted above the user may provide a. predefined background. The 
background may be captured from the capturing devices. The human opwator 
may define elements wifliin the screen as background elements. Such can be 
moving shades or areas of little interest and the like. 

5 Note should be taken that the above-described steps for the detection 

of a suspected object and the associated control parameters are exemplaiy only. 
Diverse other sequences of steps and different control parameters could be used 
in order to achieve the mherent objectives of the {nresent inventiotu 

The second preferred embodiment of the invention regards a detection 

10 of vehicles parked in restricted area or mo\ing in restricted lanes. Airports, 
government buildmgs, hotels and other institutions ^ically forbid vehicles fix>m 
parking in specific areas or. driving in restricted lanes. In some areas paridng is 
forbidden all the time while in other areas parking is allowed for a shrat period, 
such as several minutes. In the second preferred embodiment of the invdition a . 

15 system and method is proposed that detect vehicles' parking in restricted areas for 
more than a pre-defined nxmiber of time units and graerates an alarm when 
identifying an illegal parking event of a specific vehicle. In another preferred 
embodiment the system and method of the presait invention can detect whether 
persons disembark or embark a vehicle in predefined restricted zones. The use of 

20 the embodhnent described m association with.Figs. 3A, 3B can be employed in 
association with the application of the invention described below in association 
with Figs. 4A,4B. 

Referring now to Fig. 4A which shows ah exemplary flowchart 
illustrative of the operation of the city center appKcation. An iDegal parking event 

25 is detected via the performance of a sequence of operative steps. The logic 
underlying the performance is based on a specific exemplary scenario. In the 
scenario it is assumed that one or more persons may drive into a security-s^itive 
area in a specific vehicle. The vehicle could contam a powerfiil hidden explosive 
device that designed to be activated by one of flie occiq)ants of the vehicle or 

30 persons later embarking said vehicle in a restricted zone. Alternatively, the one or 
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more occupants of Ihe veUcle could be armed and may plan an armed attack 
against specific targets, such as individuals entering or exiting the building. In 
the first case scenario one of the occupants may surreptitiously (in a mmmer that 
is non-recognizable by the monitoring cameras) activate a time-del^ fijse 
5 mechanism comiected to the explosive device that is operative in the timed 
detonation of the device. Subsequently, the occupants may abandon the vehicle 
and leave the security-sensitive area for their own safety in order not to be 
exposed to the dam^e effected by the expected detonation of the ejsplosive 
device. In a second case scenario, the occupants may remam m the vducle while 
10 • waiting for the potential target, such as an individual about to enter the -scene 
either firom the building or driving into the area. In another embodiment the 
vducle may park in a restricted zone where such paridhg is not allowed in 
specific hours or whrae parking or standing is restricted for a short period of time. 
The following operative conclusions are achieved by flie suitable execution of 
15 sets of computer instructions embedded within a spedfically developed conq)Utcr 
program. The program is operative m the analysis of a sequence of video images 
or other captured data received j&om the cameras covering a security-sensitive 
area (referred herein undo: to as the video scene). An illegal paridng event is 
identified by die program when the following sequence of sub-events takes place 
20 and detected by the execution of the program: a) an object enters the video scene 
(76). The system idraitifies the object and in accordance with size and sh^e is 
assumed to be a vehicle occupied by one or more individuals, b) The object has 
subsequently to entering the restricted zone stopped moving (78). It is assumed 
that the vehicle has stopped, c) The dimensions of the object are above a pre- 
25 defined dimension parameter value (80). The dimraisions of the object are 
checked m order to distinguish between vehicles and pedestrians that m^ enter 
the same area and stop therem (for example, sitting do\wi on a bench). The size of 
the object is also unportant to deteraune the direction of movement of the object 
Thus, objects growing bigger are moving in fbe direction of ^ capturing device 
30 while objects whose size is reduced over tune are assumed to be movmg aw^ 
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from the capturing device. Certain predefined parameters as to aze may be pre- 
programmed into the system in association with a specific background image, d) 
Hie object stays without motion for a pre-defined" period (82). As described in 
association with Figs. 3A, 3B the system may detect whether an occupant of the 
5 bbject has left the object and is- now about to leave the wider area around the 
video scraie. Following the identification of the previous sub-evaits, referred to as 
specific video scene characteristics, the event will be identified by the system as a 

situation in which an illegally parked vehicle was left in the security-sensitive 
area unattended. Thus, the parked vehicle will be considered as a suspicious 
10 object Another event, which is recognized as suspicious, is where the vehicle is 
moving in an unpredicted direction or if an object, such as aperson, is leavingthe 
vdiicle and moving to an unpredictable direction. An predicted direction can be 

predefined and an unpredicted direction is the direction opposite or a durection 
which does not match a predefined direction of flow of persons or vehicles. For 

15 example, in an aijport the sidewalk where persons disembark &om vdiicles can 
be defined as ^predicted direction and the side opposite the sidewalk and across 
the lanes of travel can be defined as the unpredicted direction. Thus, if persons 
disembarking vehicles at the airport leave the vehicle unattended (stationary for a 
predefined period of time) and walk or run not towards the airport, but rather to 

20 Hie opposHe the system identifies a suspicious evrat In city centers the same can 
be applied near bus depots or train stations and also in retail establishments 
momtoring areas not for public access. 

Another parameter, which can be viewed, is the speed of the object 
25 Speeding aw^ from the vdiicle can be an additional indicator that a suspicious 
event is taking place. The parked vehicle may also be regarded as suspidous if it 
is parked in the restricted zone more than a predefined period of time. 
Consequently, the proposed system may generate, display and/or distribute an 
alarm indication. Alternatively, if the occupants of the vehicle did not leave the 
vehicle but still wait in the vehicle an alat can be raised, assuming a person is 
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waiting in the vehicle in suspicious circumstances or parking illegally. In this 
scenario too the parked vehicle will be considered a suspicious object 
Consequently, the proposed system may graerat^ display and/or distribute an 
alarm indication. Once an alarm is raised the officer reviewing the monitored 

5 scene may request the system to provide a playback so as to identify the objects in 
question. Once playback resumes the ofiBcer may tap on a touch sensitive screen 
(or select the image by other means such as a mouse, a keyboard, a light pen and 
Ifae like) and the ^stem may play bade the histoiy of video captured in 
association with the relevant object or objects. If a second object, such as a person 

10 disembarked Uie vehicle die oflScer may tap the object and request a follow up 
playback associated just with that person. The playback or play forward feature 
* allows the officer to make a real time detennination as to the objects nature 
including information stored in the database (such as parameter association with 
the object) and determine tibie next action to be taken. 

is In another embodiment an alert may be raised as soon as an object in 

the size of a vehicle as determined by the relative size-of the object as predefined 
in the system enters a restricted lane. The application concerning restricted lanes 
may check the size of the vehicles in such lanes as bus lanes wherein oidy buses 
(which are larger than vehicles) are allowed. If the object is a vehicle, i.e. smaller 

20 than a bus, an alert may be raised. The system m^ identify the vehicle and later a 
ticket may be issued to the owner of the vehicle. This application is extremely 
useful for policing restricted lanes without having a police unit on the scmt. 

In another embodiment of the invention, a database of recognized 
vehicle plate numbers can be utilized to assist in flie off line investigation and 

25 associated identification of the owner of a suspicious vehicle. The database can 
also be used to determine whether the number of the license plate is stolen or 
belongs to a suspect on a pre-sitpplied list 

Referring now to Fig. 4B Ihe exemplary control parameters of the city 
center application are shown. In order to set up the illegal parking event detection 

30 application, the usor is provided with the capability of defining the following 
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parameters: a) area or areas'within the scanned zone wherein the system will 
search for suspected objects (84), b) the dimensional limits of the detected object 
(86). The minimal dimaisional values provided in order to limit flie type of object 
as a vehicle, and c) a time out value (88) that is the amount of time that may. pass 
5 • between the point^in-time at which the object stopped moving and the point-in- 
time where an alarm will be genaated. 

Note should be taken that the above-described stqps for the detection 
of an illegally parked vehicle and the associated control parameters are exenq)laiy 
only. Diverse othrar sequences of steps and dijSarent parameters could be used in 
10 order to achieve the inherait objectives of the present invaation. 

Referring now to Fig. 5 which is a flowchart describing the operation • 
of the proposed method. The fundamental logical flow of the method is 
substantially similar for both the first preferred embodiment concerning the 
unattended luggage detection and for the second preferred embodiment 
15 concerning the detection of an illegally parked vdiicle. At stqp 93 a set of 
suspicious behavior patterns are defined. At step 95 a set of application and 
control parameters are defined. Both steps 93 and 95 are performed offline. The 
behavior patterns are implemented in the DSA application 46 of Fig. 1. vAale the 
control parameters are stored in flie application and setup parameter file 46 of 
20 Fig. 1. At step 92 video data of a pre-defined video scene is acquired by one or 
more video cameras. At step 94 flie video stream is transmitted to the video 
cq)ture unit and at step 96 the video data is sampled. Optionally tiie video data is 
recorded, compressed and archived on auxiliary storage devices, such as disks 
and/or magnetic tapes (step 98). Simultaneously the video input is transferred to 
25 the video analysis component The object detection unit analyzes the video data 
(step 100) and activates a specific DSA application 100. In the first preferred 
embodhnent tiie DSA application 100 is the unattended luggage detection while 
in tiie second preferred embodiment tiie DSA application 100 is tiie detection of 
an illegally parked vehicle. Anoflier DSA application 100 is a lost object 
30 prevention application. Oflier ^plications are evident horn tiie descrqrtioa 
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provided above in association with Figs. 3A. 3B. 4A, and 4B. At step 104 it is 
determined whether, in accordance with the specific appUcation and the 
associated parameters, including optional database parameters, an alann state was 
detected by the video analysis unit If no alarm state was identified then program 
5 control returns to the application 102. When an alarm state was raised at step 106 
the alarm event is inserted into the event database, at step 108 a suitable message 
including alarm state details is sent to Graphic User interface (GUI) control 
application, at step 1 10 an optional message mcluding alarm state-specific details 
is sent to the optional alarm distribution system to be distributed to a set of pre- 
10 defined monitor devices, sudi as wireless, personal data assistance devices, 
pagers, telephone, e-mail and the like. 

The GUI control application 108 prompts the usct for a suitable 
response concerning the alarm or optionally presents the user in real time with the 
video data sent by the camera the ouJput of which generated the alarm. The alarm 
15 can be provided as text or pop up window on the screen of the operator, as e-mail 
semt to an officer, SMS message sent to a cellular phone, an mitomated telephone 
call to an officer, a text pager message, pictures or video stream sent the offica's 
portable device or hand held device, or send via a-dry contact to generate a sirea 
or an audio or visual indication and the like. The message could be provided to 
20 one or many persons or to specific persons associated with the specific event or 
alarm. The suspidous object on the video hnages is emphasized m a graphic 
manner, such as encircling the object in a circle-like or oval graphic element that 
is overlaid on the video image. Other mfonnation concerning the object, su^ 
the object its size, speed, direction of movement, range from camera, if identified 
25 and the like, will appear next to the object's image or in another location on the 
screen. If the optional recording and archiving unit and the associated video 
archive files are implemented on the system then the user is provided with the 
option of video data re-play. "When the optional alann distribution component is 
implemented on the system, the alarm message will be appropriately distributed 
30 to a set of pre-defined and suitable pre-configured locations. 
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Objects monitored by surveDlance systems may move in unpredicted 
directions. In For example, in an airport surveillance scene a p^on may artive to 
the scene with a suitcase, enter die terminal building, leave the suitcase near the 
5 entrance of a terminal, and tiben leave the terminal. In another similar example, a 
first object (a vehicle) may arrive to the entrance of a terminal, a person (second 
object) may exit the vehicle, walk away in a direction opposite to the teraiinal, 
thus leaving the scene. In order to recognize patterns of unpredictable behavior a 
set of pre-defined rules could be implemented. These rules assist the system in 

10 capturing unpredictable behavior patterns tddng place within the scenes 

monitored by the system. 

The present system collects and saves additional information relating 
to each object An initial analysis is performed in connection witih each object 
Apart from the circle-like shape and location of the object, the system attempts to 

IS identify whether the object is a person or an inanimate object In addition, the 
object will collect and save object parameters such as the object-normalized size, 
distance from camera, color histogram. If the person is a person a face recognition 
.algorithm is activated, to try and determme whether the person is recognized. 
Recognized persons can be those persons that have been previously identified in 

20 other objects or may be faces that are provided to the system, such as &om law 
enforcement agencies or that are previously scanned by the employer. Other 
parameters m^ also be associated with the object such as name, other capturing 
devices, speed and the like. 

When su£5cient computing power is available, the system would also 

25 perform in real time a suitable analysis of the object in order to create associated 
search parameters, such as, for example, color histogram and other search 
parameters mentioned above and to immediately alert officers if the analysis leads 
to predetermined alarm status, such as when the a particular face is recognized 
whidi is a wanted perscm or a p^on not allowed or recognized in a restricted 

30 zone. In addition, in on line mode the system can identify more than one 
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parameters, such as a non-recognized face in a restricted zone and speaking in a 
foreign language, or a person not wearing a particular identifying mark (such as a 
hat or a shirt in a particular color) and the person is exiting^a vehicle. 

The proposed system and method provide real-time and ofiT-line 
5 processing of suspicious events. For example, when a vehicle arrives at a tammal 
of an airport or train station and a person leaves the vehicle to a direction opposite 
- the terminal, the present system and method will automatically alert the user. 
Such suspicious behavioral patterns are predetermined and the present system and 
method analyzes events to detect such events- The present system and method is 
10 further capable of identifying a set of linked events associated with the same 
object An object can be defined as any detected object that continues to move 
within the captured scene. An event is defined as a series of firames capturing a 
scene and. objects there within. The event can be associated with a particular 
capturing deWce. Lmked events to the same object relate to a smgle object in the 
15 same area throughout , the surveillance period whether captured by one or more 
^cameras and appearing in one or more events. The systCTOi will track (either upon 
request or automatically) an object through one or more evraits. The present 
system and method also provide the ability to associate-a-retrieved-event or object 
with unique parametCTS of such an object, in addition to ttie object oval 
20 characteristics and location. Such would include, for Sample "fac^^ 

color of clothes throu^ the use of a histogram color. Hie difference between the 
color of the clothes and the color of the shirt of an object, color per zone in the 
object, such as the color of a hair, normalized size subject to the distance ficom the 
camera, and normalized shape of objects such as the size of a suitcase. The use of 
25 object associated parameters in addition to the object's shape and position raable 
the post event data base search of an object according to the parameters to quickty 
obtain the event or events associated with the object or other objects associated 
with the object Such parametra also enable the user of the present invention to 
investigate and request the system to identify a particular object or cv&oL This 
30 enables a better retrieval of the events tod objects. The system may also, in real 
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time, associate the parameters with objects and perform rule checking to 
. detennine if the objects comply with rules that are permitted in the scene, such as 
objects are not left unattended, objects move is specific directions^ objects do not 
depart from other objects in specific locations, and the like. 

5 The additional embodiments of the present system and me&od will 

now be readily apparent to person skilled in the art. Such can include crowd 
control, people counting, an offline and online investigation tools based on the 
events stored in the database, assisting in locating lost luggage (lost prevention) 
and restricting access of persons or vehicles to certain zones. The applications are 

10 both for city centers, airports, secure locations, hospitals and Ihe like. 

It will be appreciated by persons skilled in Ihe art that the pres^ 
invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defined only by flie 
claims, whidi follow. 
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CLAIMS 

WHAT IS CLAMED IS: 

I. A method for analyzing video data, comprising: 
5 receiving a video frame; 

comparing said video frame to at least one badcground 
reference frame to locate at least one differaice; 

locating a plurality of objects to form a plurality of 
marked objects; and 
10 determining a behavior pattern for at least one object 

according to said at least one difference, wherein said 
. behavior pattaai is defined according at least one scene 
diaractertstic. 



15 2. The method of claim 1, fiirther comprising producing at 

least one updated background reference frame. 

3. The method of claim 1, wherem deterauning said at least 
one difference is performed by creating a difference frame 

20 between said video frame and said at least one background 

reference frame. 

4. The method of claim 1, further comprising finding at least 
one new object when determining said at least one 

25 difference. 

5. The method of claim 1, forlha: comprising issuing an alarm 
according to said behavior pattern. 
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6. The method as claimed m claim 1 wherein a pre-defined 
pattern of suspicious behavior comprises an object 
presenting unpredictable bdbavior. 

5 7. A system for analyzing video data comprising a plurality of 

video frames, the system comprising: 

a video frame preprocessing layer for determining a 
difference between a plurality of video frames; 

an object clustering layer for detecting a plurality 
10 of obj ects according to said difference; and 

an application layer for characterizing said plurality of 
objects according to at least one scene-characteristic. 

8. The system of claim 7, wherein said difference is 
IS determined betv^een a video firame and at least one 

reference frame, the system ftirther comprising a 
background refreshing layer for preparing at least one 
updated reference fr:ame according to the said difference. 

20 9. The system of claim 8, whsein said at least one scene 

characteristic defines a behavior pattern for at least one 
object, such that if said at least one object exhibits said 
behavior pattern, said at least one scene characteristic is 
detected. 

25 

10. The system of claim 9, wherem if said at least one scene 
characteristic is detected, an alarm is generated. 

11. The system of claim 9, wherdn said at least one scene 
30 characteristic frirth^ comprises at least one parameter for 
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detennming if said at least one object exhibits said beihavior 
pattem. 

12. A sj^stem for detecting a vehicle remaining in a restricted 
zone for at least a miniTniim period of time^ comprising: 

a video content analysis module for analyzing video data 
of the restricted zone, said video content anai^is module 
further comprising an object tracking component; and 

an apiplication layer for receiving data from said video 
content - analysis module and for detecting a vehicle 
remaining in the restricted zone for at least the minimum 
period of time, and said application layer generating an 
alarm ijpon detection. 

13. A system-for- detecting unattended luggage, bag or any 
unattended object in an area, comprising: 

a video content analysis module for analyzing video data of 
the area, said video content analysis module further 
comprising an object . tracking component and an 
application layer for receiving data from said video content 
analysis module and for detecting an unattended object, 
wherein said unattended object has not been attended in the 
area for more then a predefined period of time. 

14. A surveillance system for the detection of an alarm 
situation, the system comprising the elements of: 

a video analj^is unit for analyzing video data representing 
images of a monitored area, the video analysis unit 
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comprising an object tracking module to track the 
movements and the location of a video object; 

a detection, surveillance and alarm application for 
receiving video data analysis results &om the video analysis 
5 unit, for identifying an alarm situation and to generate an 

alarm signal; 

an events database to hold video objects, video object 
parameters and events identified by the applic^on. 

10 IS. The system of claim 14 further comprises the elem^ts of. 

an application driver to control the detection, surveillance 
and alarm application; 

a database handler to access, to update and to read the 
events database; 

IS a user interfiEice component to communicate Willi a user of 

the system; 

an application setup and control component to define the 
control parameters of the application; 
an application setup parameters table to store the control 
20 parameters of the application. 

16. Hie system as claimed in claim 14 fiirfher comprises the 
elements of: 

a video data recording and compression unit to record and 
25 compress video data representing images of a monitored 

area; 

a video archive file to hold the recorded and compressed 
video data representing images of the a monitored area; 
an alarm distribution unit to distribute the alarm signal 
30 representing an alarm situation. 
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17. The system as claimed in claim 14 further comprises the 
eleinentsof: 

at least one video camera to obtain the images of a 
5 monitored area; 

at least one a video capture component to capture video 
data representative of the images of the monitored area; 

at least one video transfer component to transfer the 
captured video "data to Ihe video analysis unit and the 
10 recording compressing and archiving unit; 

at least one computing and storage device. 

18. The system as claimed in claim 14 wherein the object 
tracking module comprises tilie elements of: 

15 a video frame preprocessing layer for determining the 

difference between at least two video frames; 

an objects clustering laya: for deteddng at least one 
objects in accordance with Ihe determined difGerence; 
a scene characterization laya: for characterizmg the a least 
20 one obj ect according to least one characteristic of a scene; 

a background refreshing layer for preparing at least 
one updated reference according to the deteimined 
diflFerence. 



25 



19. The system as claimed in claim 14 wherein the detection 
surveillance and alarm application is operative in tiie 
detection, of at least one unattended object in tiie monitored 
area. 
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20. The system as claimed in claim 17 wherein any of the at 
least one video camera; the at least one video c^turing 

. component; flie at least one video transfer conq)onent and 
the at least one computing and storage device- can be 
5 separated and can be located in differraiflocations. 

21. The system as claimed in claim 17 wherein the interface 
between the at least one video camera; flie at least one video 
capturing component; the at least one video transfer 

10 component and the at least one conq)Uting and storage 

device is a local or wide area networic or a padced-based or 
• cellular or radio frequency or micro wave or satellite 
netwodc 

15 . • 

. 22. The system as claimed in claun 19 wherein tiie at least one 

unattended object is a luggage left in an airport terminal for 
a pre-determined period. 

20 23. The system as claimed in claim 22 the at least one 

unattended object is a vehicle parking in a restricted zone 
fOT a pre-defined period. 

24. The system as claimed in clahn 14 whraein the detection 
25 surveillance and alarm application is operative in Ac 

detection of an uiq)redicted object movonent 

25. The system as claimed in claim ,14 wherein flie analysis is 
also performed on audio data or thMmal imaging data or 
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radio frequency data associated with the video data or the 
video object in synchronization with the video data. 

26. The system as claimed in claim 17 wherein the video 
capture component captures audio or thearmal inforaiation or 
radio frequency information in synchronization with the 
video data. 



10 27. A surveillance method for the detection of an alann 

situation, the surveillance to be performed on at least one 
monitored scene having at least one camera, the mediod 
comprising the steps of: 

obtaining video data from the at least one camera 

15 representing images of a the at least one monitored scene; 

- analyzing the obtained video data represoiting images 
of Ae at least one object within the at least ;one monitored 
scene, the analyzing step-comprising of idmtifjang the at 
least one object within the video data; and 

20 insertJBg the identified at least one object and tiie at least 

one cvesA into an event database. 

28. The method of claim 27 frffthw comprising the steps of: 

retrieving at least one of the object assodated with at 
25 least pne event; 

according to user instruction displaying the video 
event associated with the retrieved at least one object 

29. The method of claim 27 further comprising flie stq)s of: 
30 retrieving at least two events;. 



33. 



wo 03/067360 



PCT/IL02/01042 



associating according to. parameters of the at least one 
object, the at least one object with the at least two events. 

30. The method of claim 27 fiirlher comprising tiie steps of: 
5 debriefing the at least one object associated with the at 

least one event to identify the pattern of behavior or 
movement of the at least one object within the at least one 
scene within a predefined pmod of time. 

10 31. The method ofclaim 27 further comprising Ihe steps o£ 

^ pre-defining patterns of suspicious behavior; and 
pre-defining control parameters. 

32. The method of claim 27 further comprising tiie steps of: 
IS recognizing an alarm situation according to the pre- 

defined-patterns of suspicious behavior; and 

generating an alarm signal associated with the recognized 
alarm situatiorL 7 ~ ; 

20 33. The method as claimed in 27 furthra comprises the steps of. 

implementing patterns of suspicious behavior 
introducing pre-defined control paramet^; 
recording, compressing and archiving the obtained 
video data; 

25 distributing the alarm signal representing an alann 

situation across a pre-defined range of user devices. 

34. The method as claimed in claim 27 ^^erein a pre-defined 
pattern of suspicious behavior comprises: 
30 an object entering a monitored sc^e; 
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the object separating into a furst di^ 
distinct object in the monitored scene; 

the first -distinct object remaining in the monitored scene 
withoxit movement for a pre-defined period; and 
5 the second distinct object leaving the monitored scene. 

35. The method of claimed in claim 34 wherein the pre-defined 
pattern of suspicious behavior comprises: 
an object entering Ihe monitored scene; 
10 the object ceasmg its movanoit; 

the size of the object is recognized as being above a 
pre-defined parameter value; and 

the object remaining imm9bile for a period recognized 
as being above a pre-defined parameter value. 



15 



36. The method of claim 35 further comprising identifying 
information associated witii the object for the purpose of 
identifying the at least one object 



20 
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OBJECT ENTERS VIDEO SCENE 
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OBJECT IS SEPARATED INTO A 
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FIG. 3A 
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OBJECT ENTERS VIDEO SCENE 
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OBJECT SIZE IS ABOVE 
PRE-DEFINED SIZE PARAMETER 
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PARAMETER 



82 



FIG. 4A 
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PATTERNS OF SUSPICIOUS BEHAVIOR ARE DEFINED 
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VIDEO DATA ACQUIRED BY VIDEO CAMERAS 
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VIDEO DATA IS RECORDED 
COMPRESSED AND ARCHIVED 
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VIDEO DATA IS ANALYZED BY 
OBJECT DETECTION UNIT 
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DETECTION SURVEILLANCE ALARM (DSA) APPLICATION 




106 



NEW EVENT IS INSERTED INTO DATABASE 



I 



MESSAGE IS SENT TO GUI CONTROL APPLICATION 



MESSAGE IS SENT TO ALARM DISTRIBUTION SYSTEM 



108 



110 



FIG. 5 



